An Asymptotic Analysis of Random Partition Based Minibatch Momentum Methods for Linear Regression Models

Momentum methods have been shown to accelerate the convergence of standard gradient descent algorithm in practice and theory. In particular, minibatch-based with momentum (MGDM) are widely used solve large-scale optimization problems massive datasets. Despite success MGDM practice, their theoretical properties still underexplored. To this end, we investigate based on linear regression models. We first study numerical further provide theoretically optimal tuning parameters specification achieve faster rate. addition, explore relationship between statistical resulting estimator parameters. Based these findings, give conditions for efficiency. Finally, extensive experiments conducted verify our results.